An Asymptotic Expansion for Permutation Tests with Several Samples

Affine Invariant Multivariate Rank Tests for Several Samples

Affine invariant analogues of the two-sample Mann-Whitney-Wilcoxon rank sum test and the c-sample Kruskal-Wallis test for the multivariate location model are introduced. The definition of a multivariate (centered) rank function in the development is based on the Oja criterion function. This work extends bivariate rank methods discussed by Brown and Hettmansperger (1987a,b) and multivariate sign...

Permutation Tests for Classification

We introduce and explore an approach to estimating statistical significance of classification accuracy, which is particularly useful in scientific applications of machine learning where high dimensionality of the data and the small number of training examples render most standard convergence bounds too loose to yield a meaningful guarantee of the generalization ability of the classifier. Instea...

Comparison of Several Tests for Combining Several Independent Tests

Classifier Selection with Permutation Tests

This work presents a content-based recommender system for machine learning classifier algorithms. Given a new data set, a recommendation of what classifier is likely to perform best is made based on classifier performance over similar known data sets. This similarity is measured according to a data set characterization that includes several state-of-the-art metrics taking into account physical ...

Permutation tests for nonparametric detection

In this paper, the authors provide a methodology to design nonparametric permutation tests and, in particular, nonparametric rank tests for applications in detection. In the first part of the paper, the authors develop the optimization theory of both permutation and rank tests in the Neyman-Pearson sense; in the second part of the paper, they carry out a comparative performance analysis of the ...